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EARTH  MODEL  SELECTION  FOR  COMPUTER  SIMULATIONS 


INTRODUCTION 

The  Effectiveness  of  Navy  Electronic  Warfare  Systems  (ENEWS)  Program  in  the  Tactical  Electronic 
Warfare  Division  (TEWD)  of  the  Naval  Research  Laboratory  (NRL)  came  into  being  in  197 1  to  provide  the 
electronic  warfare  (EW)  community  the  analytical  tools  with  which  to  perform  EW  system  performance 
assessments.  To  satisfy  this  need,  ENEWS  has  generated  a  wide  variety  of  computational  tools,  and,  from 
the  beginning,  has  used  digital  computers  to  create  simulated  electromagnetic  (EM)  environments 
(scenarios).  These  environments  include  motion  for  subsurface,  surface,  and  airborne  elements  of  tactical 
engagements,  and  the  necessary  parameters  for  generating  signal  emissions.  Until  the  early  1980s,  these 
tactical  situations  covered  distances  of  not  more  than  100  to  200  nmi  and  were  placed  in  open  ocean  and 
midlatitudinal  regions.  A  flat  Earth  model  representation  has  served  adequately  for  such  scenarios. 
However,  over  the  last  6  to  10  years,  mission  requirements  had  to  include  Earth  satellites,  sea  launched 
ballistic  missiles  (SLBM),  sea  launched  cruise  missiles  (SLCM),  and  hostile  long-range  bombers  and 
reconnaissance  aircraft.  All  these  elements  can  operate  over  greater  distances  than  200  nmi  as  well  as  at 
high  latitudes  and  hence  require  the  use  of  Earth  models  that  remain  accurate  within  tolerance  over  long 
distances.  This  report  provides  a  basis  for  selecting  an  appropriate  Earth  model  that  will  allow  for  the 
inclusion  of  the  elements  already  mentioned  and  that  can  solve  future  problems. 

The  selection  of  an  appropriate  Earth  model  is  one  of  the  first  tasks  that  analysts  perform  to  construct 
computer  simulations  requiring  range  and  bearing  computations.  Frequently,  a  selection  is  made  solely  on 
the  basis  of  ease  of  implementation.  This  approach  may  yield  adequate  results  under  certain  conditions  and 
error  tolerances  as  indicated  above  in  the  ENEWS  experience.  However,  as  demonstrated  in  this  report,  the 
selection  of  a  particular  Earth  model  may  not  necessarily  yield  the  best  approach.  For  example,  the  choice 
of  one  Earth  model  applied  at  equatorial  regions  may  not  remain  within  the  required  error  budget  when  it  is 
applied  at  midlatitudinal  regions.  In  fact,  to  remain  within  certain  accuracies,  it  may  be  required  to  use 
several  Earth  models  selectively  depending  upon  location  and  distances  within  the  same  application 
program.  The  prime  intent  of  this  analysis  is  to  provide  a  method  for  selecting  an  appropriate  Earth  model 
based  on  allowable  range  errors  and  CPU  timing.  The  analysis  summary  provides  a  useful  nomogram  for 
performing  this  function. 

Specifically,  this  report  is  a  study  of  three  different  categories  of  Earth  models;  flat  Earth,  spherical 
Earth,  and  ellipsoidal  Earth.  Within  each  category,  two  or  more  approaches  are  examined  to  determine 
whether  any  significant  differences  exist  within  the  category.  CPU  utilization  analyses  are  performed  and 
included  for  completeness. 

FLAT  EARTH  MODELS 

A  typical  flat  Earth  model  uses  a  transformation  that  is  valid  for  small  areas  of  coverage.  Because  of  its 
simplicity,  it  is  computed  very  fast.  This  transformation  assumes  that  the  Earth  is  spherical  with  a 
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circumference  of  21 ,600  nmi  and  that  corresponds  to  a  distance  of  60  nmi/deg  along  any  great  circle.  For 
reference  purposes  throughout  this  report,  the  sphere  is  referred  to  as  the  Navigation  Sphere.  When  site 
locations  are  given  as  latitudes  and  longitudes,  the  positions  are  converted  to  a  flat  Earth  approximation  by 
using  these  criteria.  Details  of  this  transformation  are  summarized  here  for  ready  reference. 

The  transformation  from  the  Navigation  Sphere  to  the  flat  Earth  approximation  is  established  by 
taking  two  Points,  O  and  A,  on  the  surface  of  this  sphere.  These  points  are  defined  by  geodetic  coor¬ 
dinates  (</>!,  and  (<t>2 ,  X2)  respectively,  where  4>,  and  X,  are  the  latitude  and  longitude  respectively. 
A  mapping  of  the  difference  in  latitude  and  longitude  onto  a  flat  plane  is  given  by 

A y  =  60  A  <f> 

Ax  =  60  AX  cos  <j> 2  (1) 

where 

A</>  =  <f>2  ~  <f>i 
AX  =  X2  —  X] 

and  where  the  measurement  is  made  from  point  O  (the  observer)  to  Point  A.  By  using  this  transfor¬ 
mation,  for  point  O  on  the  plane  (x ! ,  y , )  and  differences  in  latitude  and  longitude  A0,  AX,  the  termi¬ 
nal  point  A  (x 2,  y 2)  computes  as  follows: 

yi  =  y\  +  ^ 

x2  =  *1  +  Ax  (2) 

where  Ay  and  Ax  are  given  by  Eq.  (1). 

Figure  1  shows  this  transformation,  where  point  O  is  the  observation  point  and  point  A  on  the  sphere  is 
mapped  (by  Eq.  (2))  to  point  A'  on  the  x,  y  plane.  Figure  2  shows  the  associated  bearing  relationships.  In 
both  figures  the  point  of  tangency  is  assumed  to  be  at  point  O.  From  the  preceding  formulas,  the  distance 
OA'  is  calculated  as  shown  in  Eq.  (3): 


Fig.  1  —  Distance  relationship  of  XY  flat  Earth 
and  reference  ellipsoid 
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Fig.  2  —  Bearing  relationship  of  XY  flat  Earth 
and  reference  ellipsoid 


Equation  (1)  uses  the  cosine  of  the  latitude  <t>2  a*  Point  A  to  modify  the  longitudinal  difference, 
AX.  For  minor  latitude  differences  (small  A <f>)  with  point  O  at  or  near  the  equator,  this  calculation 
may  be  satisfactory.  A  more  precise  calculation  can  be  obtained  by  taking  the  cosine  of  the  mean 
latitude.  Thus,  Ax  in  Eq.  (1)  becomes: 


Ax  =  60  AX  cos 


$  i  +  <t>  2 
2 


(4) 


This  technique  provides  a  negligible  difference  in  computer  run  time  when  compared  to  the  first 
method,  and  it  is  considered  to  be  more  accurate.  For  reference,  Eq.  (1)  will  be  called  the  End  Point  Flat 
Earth  model,  and  Eq.  (4)  and  Ay  from  Eq.  (1)  will  be  called  the  Midpoint  Flat  Earth  model. 


SPHERICAL  EARTH  MODELS 


To  obtain  the  simplest  progression  from  a  flat  Earth  model,  a  spherical  Earth  model  is  used.  In  geodetic 
computations,  several  spheres  are  commonly  used 


1.  Mean-Axis  Sphere 

2.  Equal-Surface  Sphere 

3.  Equal- Volume  Sphere 

4.  Navigation  Sphere 

5.  Spherical  Approximation 


radius  =  6370291.091  m 
radius  =  6370289.510  m 
radius  =  6370283.158  m 
radius  =  6366707.019  m 
radius  =  6371000.000  m 


The  first  three  spheres  are  discussed  in  Ref.  1.  As  previously  mentioned,  the  Navigation  Sphere 
corresponds  to  a  sphere  of  60  nmi/deg  on  any  great  circle.  Finally,  for  the  Earth’s  spherical  radius  a 
rounded-off  number  of 6, 37 1,000  m  is  sometimes  used  to  solve  geodetic  problems.  Throughout  this  report, 
this  sphere  is  referred  to  as  the  “Spherical  Approximation.” 
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ELLIPSOID  EARTH  MODELS 

The  geometric  representation  of  the  Earth  with  geodetic  accuracy  is  obtained  by  using  a  rotational 
ellipsoidal  model.  This  is  possible  because  of  the  ever-improving  ground  and  satellite  sensors  that  provide 
more  accurate  positional  and  gravitational  data  that,  in  turn,  yield  improved  values  of  associated  reference 
ellipsoid  constants.  A  rotational  ellipsoid  is  defined  by  two  constants,  the  semimajor  axis  a  (Earth’s  radius  at 
the  equator),  and  the  semiminor  axis  b  (Earth’s  radius  at  the  poles).  An  alternative  constant,  mathematically 
related  to  the  axes  [2],  is  called  the  flattening /given  by 


/  is  the  flattening, 

a  is  the  semimajor  (equatorial)  axis,  and 
b  is  the  semiminor  (polar)  axis. 

The  ellipsoid  is  normally  specified  by  giving  a  and  /;  b  is  then  readily  calculated  by  solving  for  b  in  the 
equation: 

b  =  a(l  —  f). 

For  many  Earth  models,  this  flattening  is  approximately  1/300  [3]. 

Many  ellipsoids  have  been  developed  and  used  for  more  than  a  century.  Table  1  shows  a  partial  list  of 
such  ellipsoids,  most  of  which  were  designed  to  be  used  for  specific  geographic  locations.  For  the  past  few 
decades,  attempts  have  been  focused  on  the  development  of  a  generalized  ellipsoid  model  for  the  entire 
Earth.  The  ellipsoid  used  as  a  reference  for  this  report  is  the  most  recent  one  defined  by  the  Defense 
Mapping  Agency  (DMA);  it  is  called  the  Department  of  Defense  (DoD)  World  Geodetic  System  1984,  or 
WGS-84.  Table  1  shows  the  primary  reference  ellipsoid  constants  that  define  this  ellipsoid.  The  ellipsoid 
should  provide  surface  distance  measurement  accuracies  within  a  few  meters  anywhere  on  the  Earth. 
Current  geodetic  data  generated  by  DMA,  including  digital  terrain  elevation  data  (DTED),  are  based  on  the 
WGS-84  ellipsoid.  Reference  2  provides  details  of  the  WGS-84  ellipsoid. 

Appendixes  A  and  B  provide  the  program  code  for  any  ellipsoid  defined  by  the  semimajor  and 
semiminor  axes.  The  “Inverse  Solution”  provides  a  distance  and  bearing  calculation,  given  the  latitudes 
and  longitudes  of  two  end  points.  The  “Direct  Solution”  calculates  a  second  point,  given  the  latitude  and 
longitude  of  an  initial  point,  with  the  distance  and  bearing  to  the  end  point.  The  spherical  computation  is 
available  with  either  of  these  solutions,  by  selecting  the  semimajor  and  semiminor  axes  to  be  identical 
(a  =  b). 

ANALYSIS 

Range  and  Bearing  Errors 

This  analysis  determines  the  limits  over  which  each  of  the  three  basic  Earth  models  (flat,  spherical, 
ellipsoidal)  can  be  used.  It  is  important  to  determine  acceptable  distance  and  positional  accuracies  for 
simulations  that  require  the  use  of  an  Earth  model. 

The  approach  uses  a  computer  program  that  exercises  the  ellipsoid  programs  as  listed  in  Appendixes  A 
and  B,  along  with  a  flat  Earth  model.  This  program,  computes  polar  plots  of  range  errors  and  begins  by 
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Table  1  —  Reference  Ellipsoid  Constants 


Reference 

Elipsoid 

Year 

Semimajor 
axis  (m) 
a 

Flattening 

/ 

Used  in/by 

Airy 

1830 

6377563.396 

1/299.3249646 

Great  Britain 

Everest 

1830 

6377276.345 

1/300.8017 

India 

Hough 

1830 

6378270 

1/297 

Wake-Eniwetok 

Bessel 

1841 

6377397.155 

1/299.1528128 

Japan 

Clarke 

1866 

6378206.4 

1/294.9786982 

North  America 

Clarke 

1880 

6378249.145 

1/293.465 

France,  Africia 

International 

1924 

6378388 

1/297 

Europe 

Krassowsky 

1940 

6378245 

1/298.3 

Russia 

WGS-60 

Fischer 

1960 

6378165 

1/298.3 

(Mercury) 

1960 

6378166 

1/298.3 

WGS-66 

1966 

6378145 

1/298.25 

DoD  Products 

GRS  67 

1967 

6378160 

1/298.25 

Australia,  South  America 

Fischer 

1968 

6378150 

1/298.3 

South  Asia 

WGS-72 

1972 

6378135 

1/298.26 

DoD  Products 

GRS  80 

1979 

6378137 

1/298.257222101 

North  America 

WGS-84 

1985 

6378137 

1/298.257223563 

DoD  Products 

generating  a  circle  centered  on  an  observation  point  with  a  predetermined  range.  The  WGS-84  Direct 
Solution  calculates  the  latitudes  and  longitudes  of  points  on  that  circle,  in  1°  steps.  Then  the  flat  Earth  and 
spherical  Earth  models  are  exercised  with  their  associated  inverse  programs  to  calculate  the  distance 
(range).  The  calculated  ranges  are  then  compared  to  the  predetermined  range  to  obtain  a  range  error.  The 
range  error  assumes  the  WGS-84  ellipsoid  as  a  reference.  Bearing  errors  are  also  calculated  by  comparing 
the  calculated  bearing  with  the  input  azimuth  angle  used  to  generate  the  data. 

Flat  Earth  Models 

The  analysis  compares  the  two  flat  Earth  models  described  earlier.  Polar  plots  of  the  range  and  bearing 
errors  have  been  generated  for  various  ranges.  The  program  steps  in  preparing  the  plots  are  as  follows: 

1 .  Select  the  range  at  which  errors  will  be  calculated. 

2.  Select  the  observer  latitude  and  longitude. 

3.  Loop  through  360°  in  1°  steps. 

4.  Use  the  WGS-84  Direct  Solution  to  determine  the  latitudes  and  longitudes  of  points  on  the  range 
circle  in  1°  increments. 

5.  Use  the  two  flat  Earth  inverse  (unctions  to  calculate  new  ranges  and  bearings,  given  the  latitudes 
and  longitudes  from  steps  2  and  4. 

6.  Compare  the  newly  calculated  ranges  with  the  selected  range  from  step  1  for  range  errors. 

7.  Compare  the  newly  calculated  bearings  with  the  azimuths  defined  in  step  3  for  bearing  errors. 

Figures  3  and  4  show  the  range  and  bearing  errors  incurred  at  45  °N  0°E  for  a  distance  of  100  nmi. 
These  and  other  figures  are  presented  in  detailed  analyses  in  Appendix  D.  Figure  3  shows  the  improvement 
gained  by  using  the  Midpoint  Flat  Earth  model  instead  of  the  End  Point  Flat  Earth  model. 
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Spherical  Earth  Models 

The  next  step  is  to  select  an  appropriate  spherical  Earth  model.  The  flat  Earth  programming  steps 
discussed  earlier  are  used  again.  Spherical  models  are  substituted  in  step  5;  they  are  those  used  in  the 
ellipsoidal  solutions  of  Appendixes  A  and  B,  with  equal  radius  values  for  the  semimajor  and  semiminor 
axes.  The  five  spheres  already  described  are  reduced  to  three  spheres,  because  the  first  three  in  the  list  are, 
for  practical  purposes,  indistinguishable.  Those  chosen  for  further  study  are  the  Equal  Volume,  the 
Spherical  Approximation,  and  the  Navigation  Sphere. 

Figure  5  shows  a  graph  of  the  selected  three  spherical  models  at  500  nmi  from  the  observation  point 
45°N  0°E.  At  this  latitude  and  distance  the  Spherical  Approximation  model  is  predominantly  better  than 
either  of  the  other  two  spherical  Earth  models.  Appendix  E  furnishes  additional  comparisons  at  different 
latitudes  and  ranges. 


Comparison  of  Flat  Earth  and  Spherical  Earth  Models 

The  analysis  now  determines  the  appropriate  domains  over  which  flat,  spherical,  and  ellipsoidal  Earth 
models  should  be  used.  It  is  clear,  from  the  preceding  analysis,  that  depending  on  the  application  and 
allowable  error  tolerances,  sufficient  choices  of  models  exist  that  can  be  used.  However,  it  is  more  difficult 
to  see  when  the  adequacy  of  a  model  stops.  This  section  of  the  report  attempts  to  focus  on  this  question  solely 
on  the  basis  of  range  error  analyses. 

We  reexamine  the  two  flat  Earth  models  previously  discussed.  By  generating  data  as  shown  in  Fig.  3, 
but  for  a  large  number  of  radial  ranges  up  to  500  nmi,  we  obtain  the  maximum  range  error  incurred  at  each 
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of  the  radial  distances.  Thus,  Fig.  6  illustrates  the  error  incurred  by  the  End  Point  Flat  Earth  model.  Similar 
calculations  are  also  performed  by  using  the  Midpoint  Flat  Earth  and  Spherical  Approximation  models  and 
are  placed  on  the  same  graph  shown  in  Fig.  6.  Appendix  F  shows  similar  graphs  computed  at  other 
latitudes. 

The  analysis  now  proceeds  to  determine  the  error  contours  for  the  Midpoint  Flat  Earth  model  as  a 
function  of  latitude.  By  generating  the  same  type  of  data  as  shown  in  Fig.  3,  but  taken  at  1 0  increments  of 
observer  latitude,  it  is  possible  to  determine  the  range  distances  for  a  1-nmi  error,  2-nmi  error,  etc.  This 
produces  the  contours  that  are  shown  in  Figs.  7  through  10.  Figure  7  shows  the  End  Point  Flat  Earth  model 
contours,  and  Fig.  8  shows  the  Midpoint  Flat  Earth  model  contours.  Figures  FI  through  F7  in  Appendix  F 
compare  the  behavior  of  the  End  Point  Flat  Earth  model  to  the  Midpoint  Flat  Earth  model  at  six  selected 
latitudes.  Figures  7  and  8  provide  the  expansion  of  these  data  from  the  equator  to  the  north  polar  region.  The 
1-nmi  error  contour  shows  the  distance  to  which  one  can  go  to  incur  not  more  than  a  1-nmi  error  at  any 
latitude  north  of  the  equator.  Thus,  for  example,  in  Fig.  7,  if  an  observer  is  at  40°N  and  if  the  distance 
between  the  observer  and  a  second  point  is  less  than  or  equal  to  125  nmi,  distances  can  be  computed 
accurately  to  within  1  nmi.  Using  the  Midpoint  Flat  Earth  model  (Fig.  8)  at  the  same  latitude,  the  1-nmi 
tolerance  extends  to  approximately  360  nmi.  A  direct  comparison  of  all  the  contours  in  Fig.  7  and  8  reveals 
that  the  Midpoint  Flat  Earth  model  is  superior  to  the  End  Point  Flat  Earth  model  in  error  tolerance  at  all 
latitudes  over  all  surface  ranges.  It  is  clear,  therefore;  that,  since  the  computational  requirements  for  these 
two  models  are  equivalent  (Eqs.  (1)  and  (4)),  the  Midpoint  Flat  Earth  model  is  preferable. 


Fig.  6  —  Surface  range  errors  vs  surface  range 
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Fig.  7  —  End  Point  Flat  Earth  range  error  contours  vs  observer  latitude 


Fig.  8  —  Midpoint  Flat  Earth  range  error  contours  vs  observer  latitude 
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SphericaJ  Approximation  range  error  contours  vs  observer  latitude 


Fig.  10  —  Spherical  Approximation  range  error  contours  vs  observer  latitude 
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Having  established  the  superiority  of  the  Midpoint  Flat  Earth  model,  we  now  question  in  which  regions 
it  is  safe  to  use  this  model  before  using  a  spherical  model  such  as  the  Spherical  Approximation  model. 
Figure  9  shows  the  Spherical  Approximation  model  contours.  The  1-nmi  error  contour  shows  the  distance 
to  which  one  can  go  to  incur  not  more  than  a  1-nmi  error  at  any  latitude  north  of  the  equator.  Thus,  for 
example,  if  an  observer  is  at  15°N  and  if  the  distance  between  the  observer  and  a  second  point  is  less  than  or 
equal  to  200  nmi,  distances  can  be  computed  accurately  to  within  1-nmi.  A  2-nmi  error  is  incurred  at  15°N 
at  distances  up  to  approximately  3S0  nmi,  and  so  forth.  The  contours  of  Fig.  9  indicate  that  the  Spherical 
Approximation  to  the  WGS-84  ellipsoid  is  best  in  the  region  from  38  °N  to  about  42  °N.  Comparison  of  the 
contours  in  Fig.  8  and  those  in  Fig.  9  are  now  performed  to  determine  the  regions  where  switching  from  the 
Midpoint  Flat  Earth  model  to  Spherical  Approximation  model  should  take  place.  By  comparing  the  1-nmi 
error  contour  on  each  figure,  we  can  see  that  the  1-nmi  error  contour  in  Fig.  9  begins  to  allow  for  greater 
distance  calculations  at  36° N  and  375  nmi.  Essentially,  this  is  the  crossover  point  where  the  Spherical 
Approximation  model  yields  an  improved  representation.  Similar  comparisons  for  the  other  four  contours 
yield  the  following  approximate  crossover  points: 


Surface  Error 
(nmi) 

Latitude 

(°N) 

Distance 

(nmi) 

1 

36 

375 

2 

65 

500 

3 

39 

1050 

4 

35 

1125 

5 

29 

1250 

Below  29°N  latitude  there  appears  to  be  a  0.5-nmi  difference  between  the  two  models  in  favor  of  the 
Midpoint  Flat  Earth  model.  Beyond  that,  the  Spherical  Approximation  can  maintain  a  better  approximation 
to  the  WGS-84  ellipsoid  model. 

Figure  8  should  not  be  interpreted  to  mean  that  a  flat  Earth  model  is  adequate  over  distances  of  1000 
nmi.  In  practice,  however,  most  applications  should  hold  the  differences  to  under  1  nmi.  This  indicates  that 
a  distance  of  200  nmi  is  the  practical  extent  over  which  calculations  can  be  performed  with  such  a 
representation.  Since  the  Spherical  Approximation  model  yields  approximately  equivalent  errors  up  to 
about  80°N  (beyond  which  this  model  is  superior  to  the  Midpoint  Flat  Earth  model),  considerations  to  use 
the  Spherical  Approximation  at  all  latitudes  for  distances  of  less  than  200  nmi  are  strong.  This  trade-off, 
however,  is  performed  based  on  timing  considerations  between  the  two  approaches.  Finally,  Fig.  10  shows 
a  fine-grain  analysis  (under  1-nmi  error).  A  1000-m  error  will  not  be  exceeded  by  not  exceeding  distances 
of  220  nmi  (that  is,  approximately  1  part  in  440).  This  then  yields  a  practical  limitation  on  the  regions  over 
which  the  Midpoint  Flat  Earth  model  could  safely  be  used  below  80°N.  Above  80°N  no  flat  Earth 
representation  is  adequate. 

Comparison  of  Spherical  Approximation  Model  and  WGS-84  Ellipsoidal  Model 

By  setting  an  appropriate  error  budget,  plots  as  shown  in  Figs.  9  or  10  can  be  used  to  determine  how  far 
in  range  we  can  use  the  Spherical  Approximation  model  before  we  must  use  the  WGS-84  ellipsoidal 
calculations.  For  example,  if  an  application  program  is  used  in  the  region  between  50°N  and  60°N  with  a 
maximum  error  tolerance  of  2  nmi,  we  can  safely  apply  the  Spherical  Approximation  model  for  calculating 
distances  up  to  550  nmi.  Beyond  this  distance,  we  should  switch  to  the  WGS-84  ellipsoidal  calculations. 
For  1-nmi  error  tolerances  between  50°N  and  60°N  the  maximum  range  distance  is  about  270  nmi. 
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In  summary,  many  areas  of  interest  lie  in  the  region  between  40° N  and  70° N.  Over  this  region  the 
Midpoint  Flat  Earth  model  yields  acceptable  results  (errors  less  than  1000  m)  at  distances  under  100  nmi. 
From  100  nmi  to  approximately  250  nmi  both  the  Midpoint  Flat  Earth  and  the  Spherical  Approximation 
models  yield  similar  results.  For  the  distances  beyond  250  nmi  it  is  best  to  use  WGS-84  calculations.  For 
example,  at  50°N  both  models  yield  an  error  of  1  nmi  at  a  distance  of  approximately  270  nmi.  At  a  distance 
of  1000  nmi  the  Midpoint  Flat  Earth  model  yields  an  error  of  approximately  6  nmi  (Fig.  8)  and  the 
Spherical  Approximation  model  gives  a  3-nmi  error  (Fig.  9)  at  50° N  latitude.  Both  results  are  generally 
beyond  acceptable  limits. 

Timing  Considerations 

Examination  of  the  code  of  the  Geodetic  Inverse  solution  in  Appendix  A  shows  that  an  iterative  loop  is 
used  to  perform  the  ellipsoidal  calculation.  Subsequent  data  in  this  report  show  that  this  loop  is  repeated  only 
if  the  semimajor  and  semiminor  axes  are  not  equal;  for  a  sphere,  only  one  pass  is  made  through  the  loop. 
When  this  routine  is  used  for  ellipsoids  in  production  applications,  this  loop  can  potentially  be  time 
consuming.  A  driver  program  was  written  to  test  the  loop  and  the  associated  loop  tolerance.  The  observer  at 
point  O  is  placed  at  0°  longitude  and  0°  latitude.  The  driver  program  contains  three  loops  to  vary  the 
tolerance,  the  longitude  of  the  target  at  point  A,  and  the  latitude  of  the  target  at  point  A.  The  longitude  of 
point  A  is  changed  in  1  °  steps  from  —  1 79°  to  +1 80° .  The  latitude  of  point  A  is  varied  from  0°  to  90°  in  1 0 
steps.  Since  the  convergence  tolerance  is  directly  related  to  CPU  utilization,  the  loop  tolerance  is  varied 
from  10_1  to  10~12.  The  tolerance  of  1012  is  the  value  used  by  DMA  for  applications  that  require  high  levels 
of  accuracy.  It  is  used  here  as  a  reference  to  calculate  maximum  or  worst  case  range  measurement  errors  at 
the  other  tolerances.  All  range-  and  bearing-error  analyses  presented  earlier  in  this  report  use  the  10~12 
tolerance  for  the  WGS-84  reference  ellipsoid. 

Table  2  gives  a  summary  of  the  test  results  for  the  32760  positional  data  points  calculated.  The  A 
distance  errors  vary  approximately  an  order  of  magnitude  as  the  tolerance  varies  an  order  of  magnitude. 


Table  2  -  Maximum  Errors  from  10' 12  Tolerance 


Tolerance 

A  Distance 
(nmi) 

A  Bearing 
(deg) 

10~n 

0.00000003 

0.00000001 

10-io 

0.00000034 

0.00000014 

10'9 

0.00000302 

0.00000124 

10'8 

0.00003398 

0.00001044 

10'7 

0.00034435 

0.00008813 

10'6 

0.00337846 

0.00100208 

10'5 

0.03439104 

0.00628367 

10^ 

0.29504115 

0.15386889 

10'3 

3.42600647 

1.30084743 

10'2 

34.26006474 

11.30886079 

10'1 

36.07383288 

180.00000000 

Examination  of  the  Geodetic  Inverse  code  in  Appendix  A  shows  that  an  automatic  limit  of  100  loops  is 
set  on  each  call  to  the  routine.  This  limit  was  added  as  a  precaution  against  cases  that  converge  too  slowly. 
The  number  of  loops  actua'ly  observed  in  the  testing  gives  an  approximation  of  the  CPU's  utilization 
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required.  Table  3  shows  a  profile  of  the  number  of  loops  for  each  tolerance  magnitude  for  nonspherica! 
calculations. 

Table  3  can  also  be  used  to  generate  probability  curves  of  convergence  loop  occurrences.  From  these 
probabilities  we  can  obtain  the  associated  cumulative  probabilities  that  the  number  of  loops  incurred  is  less 
than  a  fixed  number.  Figure  1 1  shows  a  plot  of  these  cumulative  probabilities  for  four  of  the  tolerances 
tested.  The  probability  of  incurring  three  loops  or  less  with  a  tolerance  of  10~7  is  approximately  0.85.  That 
is,  85%  of  the  time  an  ellipsoidal  calculation  is  performed,  it  requires  three  or  fewer  loops  at  that  tolerance. 
However,  to  use  the  software  as  DMA  requires  for  its  purposes  (that  is,  using  a  tolerance  of  1012),  85%  of 
the  time  requires  the  use  of  five  loops  or  fewer.  Hence  it  requires  twice  the  amount  of  looping  to  improve  the 
results  by  a  tolerance  of  1 0‘5. 


Table  3  —  Inverse  Loop  Count  vs  Tolerance  Magnitude  for  WGS-84  Ellipsoid 


1  Time 

2  Times 

3  Times 

4  Times 

5  Times 

6  Times 

7  Times 

8  Times 

9  Times 

10  Times 

1 1  Times 

12  Times 

13  Times 

14  Times 

15  Times 

16  Times 

17  Times 

18  Times 
21  Times 
23  Times 

101  Times 


A  comparison  of  the  data  in  Table  2  with  the  contour  data  in  Fig.  9  indicates  that  using  a  convergence 
tolerance  of  10  4  for  the  WGS-84  ellipsoid  is  equivalent  to  using  the  Spherical  Approximation  for  surface 
ranges  under  250  nmi.  Table  3  shows  that  the  loop  count  for  a  tolerance  of  10  4  is  predominantly  less  than  or 
equal  to  2.  Hence  for  applications  requiring  a  range  error  budget  under  1  nmi.  a  tolerance  of  10  4  in  the 
ellipsoidal  model  serves  very  well. 


Data  collection  was  performed  to  obtain  WGS-84  reference  ellipsoid  timing  \  s  the  number  of  required 
loops  sampled  at  a  loop  convergence  tolerance  of  10' i:.  Table  4  summarizes  the  average  CPU  time  on  a 
VAX  8650  for  the  Inverse  and  the  Direct  Solutions  (per  call  to  each)  as  a  function  of  the  number  of  loops 
each  algorithm  takes.  Only  five  sets  of  data  were  obtained  for  the  Direct  Solution.  This  is  due  to  the  low 
probability  of  loop  counts  exceeding  5  for  that  routine  (that  is,  higher  loop  counts  could  not  be  found). 
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Cumulative  Probability 


Fig.  1 1  —  Loop  count  probability 


Table  4  -  Sample  Loop  Count  Timing  for 
WGS-84  Reference  Ellipsoid  Routines 
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An  associated  study  of  timing  to  achieve  the  number  of  computations  discussed  in  the  previous  section 
is  now  examined.  1°  steps  in  the  northern  hemisphere  are  selected  to  exercise  the  geodetic  subroutines, 
including  one  that  calculates  the  chordal  distance  (Appendix  C),  32,760  times.  For  ihe  other  geodetic 
routines,  the  observer  is  placed  at  seven  latitudes,  from  0°  to  90°  in  15°  increments,  generating  229,320 
data  points. 

VAX  8650  CPU  time  is  extracted  for  each  subroutine  call,  and  Table  5  shows  the  results.  Because  the 
routines  are  exercised  at  a  uniform  distribution  ( 1 0  steps)  in  the  northern  hemisphere,  these  numbers  may  be 
considered  good  statistical  averages. 


Table  5  -  CPU  Time  for  Each 
Geodetic  Calculation 


Geodetic  Calculation 

Per  Call 
(ms) 

DMA  Chord  Subroutine 

294.4 

End  Point  Flat  Earth  Direct 

181.0 

End  Point  Flat  Earth  Inverse 

411.6 

Midpoint  Flat  Earth  Inverse 

200.0 

Midpoint  Flat  Earth  Direct 

184.0 

Spherical  Inverse  Solution 

420.5 

Spherical  Direct  Solution 

436.3 

The  DMA  Chord  subroutine  requires  the  least  amount  of  time,  294.4  fis  per  call.  It  is  noteworthy  that 
three  inverse  calculations,  two  for  flat  Earth  and  one  for  the  spherical  Earth,  take  approximately  the  same 
amount  of  time.  This  shows  that  replacing  a  flat  Earth  model  with  a  spherical  Earth  model  has  a  minimal 
impact  on  CPU  time. 

Table  6  presents  the  CPU  requirements  for  the  WGS-84  ellipsoid  loop  convergence  tolerances  ranging 
from  10_  12  through  10'1 .  The  CPU  utilization  drops  dramatically  as  the  convergence  tolerance  is  lessened. 
Table  3  shows  the  reduction  in  convergence  looping  as  a  result  of  this  tolerance  reduction.  Table  4  shows 
the  corresponding  reduction  in  CPU  time  as  a  function  of  the  number  of  loops.  Thus  we  can  readily  predict 
the  resulting  tendencies  as  shown  in  Table  6. 

Using  all  of  the  aforementioned  tables  and  figures,  we  can  now  perform  an  Earth  model  selection 
process.  As  an  example,  the  ENEWS  Program  generates  digital  computer  representations  of  large  scale  EM 
environments  with  ship,  aircraft,  and  missile  motion.  These  representations,  referred  to  as  scenarios, 
currently  extend  over  distances  of  up  to  800  nmi.  Ships,  missiles,  and  aircraft  are  considered  as  mass  points; 
their  dimensionality  is  considered  only  when  radar  cross  section  is  calculated.  An  assumption  of  an  error 
budget  of  under  1  nmi  (a  distance  of  the  order  of  magnitude  of  approximately  six  times  the  length  of  a  Navy 
carrier)  leads  to  the  requirement  of  using  a  convergence  loop  tolerance  of  10-4  or  less  (Table  2)  when  using 
the  WGS-84  reference  ellipsoid  calculations.  Since,  in  scenario  motion  computations,  the  Direct  Geodetic 
Solution  is  the  one  predominantly  used,  the  discussion  focuses  only  on  values  for  this  computation.  Table  7 
is  extracted  from  Tables  2,  5,  and  6  and  Fig.  9. 
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Table  6  —  CPU  Time  Per  Call  for 
Each  Convergence  Tolerance  for  the 
WGS-84  Reference  Ellipsoid 


Convergence 

Tolerance 

Inverse 

(as) 

Direct 

(as) 

— w™ 

747.2 

697.9 

10~ 11 

693.8 

633.7 

io~ «° 

662.0 

627.1 

10-09 

651.1 

591.6 

10-08 

610.0 

574.9 

10-07 

578.7 

569.8 

10-06 

551.3 

523.2 

10-05 

517.7 

510.9 

10-O* 

495.8 

506.1 

10-03 

464.2 

455.5 

10-02 

405.0 

424.6 

10-01 

413.0 

428.9 

Table  7  —  Timing  and  Error  Considerations  for  Spherical 
Approximation  and  Selected  WGS-84  Loop  Tolerances 


CPU  Time 
per  call 

(as) 

Maximum  Distance  Error 
from  WGS-84  Ellipsoid 
(10~12)  at  800  nmi 

(m) 

(nmi) 

Spherical  Approximation 

436.3 

7408 

WGS-84  (10“4) 

506.1 

546 

WGS-84  (10~5) 

510.9 

63 

The  incurred  Spherical  Approximation  model  distance  error  is  taken  over  all  latitudes.  Restricting  the 
error  to  latitudes  between  40°N  and  70°N  still  yields  an  error  of  approximately  3  nmi  at  a  surface  range  of 
800  nmi.  This  range  measurement  error  far  exceeds  the  allotted  error  budget.  Figure  9  shows  that  it  is 
impossible  to  achieve  the  desired  error  budget  under  1  nmi  at  distances  of  800  nmi.  Thus,  this  narrows  the 
model  selection  choice  to  the  use  of  the  WGS-84  calculations  with  a  13.5  factor  improvement  in  the 
resultant  error  (0.295  nmi  vs  4  nmi)  at  a  cost  of  only  70  as  more  per  subroutine  call.  The  choice  of  a 
tolerance  of  10'5  rather  than  10"4  yields  an  improvement  factor  of  approximately  118  times  (0.034-nmi 
error  vs  4  nmi)  at  a  cost  of  only  4  pis  more  per  call  than  at  the  10"4  tolerance.  It  is  clear  that  the  most  cost 
effective  choice  of  Earth  model  to  use  for  the  scenario  application  described  above  is  the  WGS-84  reference 
ellipsoid  model  that  uses  a  convergence  loop  tolerance  of  10~5.  For  scenarios  covering  smaller  distances, 
either  a  larger  tolerance  for  the  WGS-84  reference  ellipsoid  or  the  Spherical  Approximation  model  may  be 
used.  Thus,  for  example,  for  a  250-nmi  coverage  (see  Fig.  9)  a  1-nmi  error  budget  is  achieved  at  a  cost 
reduction  of  506.1  —  436.3  =  69.8  ^s  per  call  (Table  7). 
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Summary 

To  summarize  the  Earth  model  selection  process  in  a  general  manner,  a  nomogram  of  the  distance 
coverage,  error  budget,  and  additional  CPU  timing  costs  is  constructed  for  the  latitudesO°N  through  70°N. 
The  latitudes  are  separated  into  two  belts,  the  equatorial  region  (0°N  to  40° N)  and  the  midlatitudes  (40° N 
to  70°N).  This  is  accomplished  by  constructing  Table  8  from  the  preceding  tables  and  figures.  Timing  costs 
shown  in  units  of  microseconds  are  CPU  time  in  addition  to  the  time  required  to  compute  the  direct  Midpoint 
Flat  Earth  (MPFE)  model  calculation.  The  distances  in  the  table  are  the  maximum  distance  computed  under 
the  constraint  of  the  associated  maximum  error.  For  example,  the  Spherical  Approximation  (SA)  model  can 
be  used  up  to  a  distance  of  250  nmi  with  a  maximal  error  of  1  nmi  in  the  midlatitude  regions  at  a  cost  of 
252  fis  above  the  time  required  for  an  MPFE  direct  calculation.  Figure  12  is  then  constructed  from  the  data 
in  Table  8.  The  nomogram  is  constructed  for  the  purpose  of  serving  as  an  Earth  model  selection  tool.  It 
provides  the  means  for  determining  possible  distance  coverages  within  specified  range  error  budgets  with 
associated  CPU  utilization  costs  that  are  additional  to  those  incurred  by  using  the  MPFE  model.  To  gain  an 
understanding  of  how  to  use  this  nomogram,  the  reader  must  work  through  the  inferences  drawn  from  it  in 
the  next  paragraph. 

From  the  nomogram,  with  the  WGS-84  reference  ellipsoid  represented  by  all  the  shaded  regions,  we 
can  infer  the  following: 

1.  The  SA  model  cannot  be  used  to  compute  a  distance  of  more  than  250  nmi  with  a  1-nmi  error 
budget;  nor  can  the  MPFE  model.  Only  the  WGS-84  operating  at  a  convergence  tolerance  of  10‘2 3 
or  less  can  be  used. 

2.  The  MPFE  model  actually  outperforms  the  SA  model  (as  previously  discussed)  in  the  equatorial 
region.  However,  it  is  limited  to  a  distance  coverage  of  720  nmi  for  error  budgets  under  3.42  nmi. 
For  an  additional  cost  of  271  /ns,  this  limitation  is  alleviated  by  the  use  of  the  WGS-84  model  at 
10“ 3  convergence  loop  tolerance.  Within  an  error  budget  of 0.295  nmi,  a  switch  must  be  made  to  a 
tolerance  of  10"4  at  a  cost  of  an  additional  51  /ns  per  subroutine  call. 

3.  For  distances  greater  than  500  nmi  and  error  budget  greater  than  3.42  nmi,  the  SA  model  is 
superior  to  the  MPFE  model  at  midlatitudes.  However,  the  WGS-84  reference  ellipsoid  model 
outperforms  the  SA  model  by  11  /ns  per  call  with  no  distance  limitation.  Thus,  for  these  error 
budgets,  the  SA  model  is  not  cost  effective  in  comparison  to  the  WGS-84  model. 

4.  For  budget  tolerances  under  0.295  nmi  there  is  very  little  difference  in  coverage  between  the  SA 
model  and  the  MPFE  model,  with  an  attendant  maximum  coverage  of  75  nmi.  The  WGS-84 
reference  ellipsoid  model  operating  at  10-4  convergence  loop  tolerance  provides  an  unlimited 
coverage  at  an  additional  CPU  time  cost  of  322  /xs. 

APPLICATION  TO  ENEWS  SIMULATIONS 

By  necessity,  ENEWS  must  be  responsive  in  performing  EW  assessments  because  of  more  complex 
and  changing  technologies.  Such  technologies  include  the  use  of  satellites,  enhanced  interferometry,  as  well 
as  improvements  in  EW  systems  due  to  the  extensive  integration  of  on  board  computers  and 
microprocessors  as  part  of  these  systems.  The  Earth  models  must  be  chosen  so  that  they  serve  as  adequate 
vehicles  with  which  to  conduct  studies,  the  results  of  which  fall  within  acceptable  error-tolerance  levels. 
Clearly,  the  inclusion  of  dynamic  Earth  satellites  precludes  the  use  of  any  flat  Earth  models. 
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Table  8  —  Distance  Coverages  and  Additional  CPU  Timing  Costs 


Fig.  12  —  Distance  coverage  and  additional  CPU  timing  costs 


The  selection  process  of  an  Earth  model  for  current  and  future  applications  within  the  ENEWS 
Program  rests  between  the  use  of  a  spherical  Earth  model,  such  as  the  Spherical  Approximation  model,  or 
an  ellipsoid  of  revolution,  such  as  the  DoD  standard  World  Geodetic  System.  Since  the  CPU  timing 
between  the  two  methods  is  effectively  equivalent,  timing  does  not  contribute  to  the  decision  process.  The 
remaining  factors  are  accuracy  and  computer  programming  considerations. 

First  we  address  the  question  of  accuracy  and  data  sources.  As  this  report  shows,  accuracies  of  at  best 
3.5  to  4  nmi  can  be  achieved  for  distances  at  and  beyond  800  nmi  by  using  the  Spherical  Approximation 
model.  At  the  same  distance,  the  WGS  approach  yields  accuracies  that  are  less  than  0.3  nmi  from  the  DoD 
standard.  This  is  a  significant  difference  for  applications  that  require  the  placement  of  surface  to  air  missile 
(SAM)  sites  and  any  other  target  elements.  Generally,  ENEWS  receives  site  data  from  sources  such  as  the 
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Electronic  Order  of  Battle  (EOB)  and  the  Air  Order  of  Battle  (AOB).  These  data  are  obtained  primarily 
from  maps  that  use  local  datums  (that  is,  local  national  surveys).  If  local  surveys  are  not  available,  point 
positions  are  drawn  from  the  WGS  system.  Thus,  these  local  datums  are  generally  not  consistent  with  each 
other.  For  problems  covering  transoceanic  or  even  intercontinental  distances,  this  problem  is  very 
apparent.  The  WGS  standard  provides  the  transformation  to  convert  specific  local  coordinates  into  the 
WGS  system  and  vice  versa  [2].  However,  the  differences  in  point  location  between  using  local  datums  and 
WGS-84  varies  generally  from  200  m  to  1  or  2  nmi.  Consequently,  WGS-84  provides  a  consistent  model 
that  covers  large  distances  with  site  coordinate  data  and  with  known  maximal  variations  from  the  actual 
location.  A  spherical  Earth  model  does  not  ensure  this  consistency  over  more  than  a  single  datum. 

For  the  problems  that  involve  ground  based  radars,  local  terrain  elevation  features  play  an  essential 
role  in  determining  masking  effects,  that,  in  turn,  affect  the  contributions  to  observable  pulse  densities  in 
EM  environments.  DMA  provides  Digital  Terrain  Elevation  Data  (DTED)  for  most  land  masses.  These 
data  are  referenced  to  the  WGS-84  system.  ENEWS  has  developed  software  to  retrieve  and  process  DTED 
[4]  and  currently  maintains  a  database  covering  regions  of  interest.  Also,  many  vendor-supplied 
mission-planning-graphics  software  tools  use  DTED  for  determining  elevations  and  line  of  sight. 
Attempting  to  use  DTED  for  such  analyses  with  spherical  Earth  models  causes  a  misplacement  of 
geographic  features  by  several  miles.  This  would  result  in  distortions  of  resultant  pulse  densities  attributable 
to  many  ground  sites  as  viewed  from  long  distances. 

Finally,  from  the  programming  consideration  viewpoint,  it  is  important  to  achieve  maximum 
flexibility  to  be  able  to  solve  future  problems  while  incurring  minimum  impact  in  programming 
implementation.  It  is  a  fact  that  the  Direct  and  Inverse  Solution  software  implementation  is  identical  for  the 
Spherical  Approximation  model  as  well  as  for  the  WGS-84  model.  Since  the  Spherical  Approximation 
model,  or  for  that  matter  any  spherical  Earth  model,  is  valid  only  in  a  local  sense  (that  is,  over  limited 
number  of  contiguous  datums),  these  do  not  offer  the  desired  flexibility  needed  in  future  growth  in  solving 
EW  problems. 

CONCLUSIONS 

The  following  conclusions  may  be  drawn  from  the  previous  analyses: 

1 .  End  Point  Flat  Earth  models  are  inadequate  representations  of  the  Earth  even  over  short  ranges 
(less  than  100  nmi.)  However,  other  flat  Earth  models  such  as  the  Midpoint  Flat  Earth  model  may 
be  adequate  for  many  applications. 

2.  Flat  Earth  models  should  not  be  used  to  calculate  distances  greater  than  100  nmi,  especially  north 
of  40°N  latitude  if  attempting  to  achieve  less  than  1000-m  accuracies. 

3.  The  Spherical  Approximation  model,  on  the  average,  yields  an  equivalent  if  not  better 
representation  than  the  Midpoint  Flat  Earth  model  over  most  of  the  globe.  This  type  of  model  can 
serve  adequately  for  distances  up  to  approximately  250  nmi  in  the  40°N  to  70°N  regions  while 
incurring  differences  less  than  1000  m  with  respect  to  the  WGS-84  ellipsoid. 

4.  Beyond  previously  indicated  ranges,  the  WGS-84  model  should  be  used.  Data  indicate  that  four 
convergence  loops  or  fewer  would  be  incurred  approximately  95%  of  the  time  that  Geodedic 
Inverse  Solutions  are  performed.  On  the  average,  one  call  for  a  Geodedic  Inverse  Solution 
requires  0.5  ms  on  a  VAX  8650.  The  Direct  Solution  requires,  on  the  average,  0.363  ms. 
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For  the  accuracies  achieved,  and,  in  view  of  the  constant  improvement  in  the  CPU  throughput  of 
new  machines,  the  CPU  utilization  is  a  minimal  and  decreasing  parameter  in  the  Earth  model 
selection  process.  Tolerances  can  be  “tightened”  if  necessary  with  the  arrival  of  faster  digital 
computers. 

5.  For  accuracy  requirements  under  1  nmi,  the  WGS-84  reference  ellipsoid  used  with  a 
convergence  loop  tolerance  of  10“5  over  all  ranges  on  a  hemisphere  is  the  most  cost  effective 
Earth  model  selection  for  ENEWS  scenario  applications. 

RECOMMENDATIONS 

As  a  result  of  the  preceding  analyses  and  conclusions,  it  is  evident  that  the  choice  of  an  Earth  model 
based  on  allowable  range  errors  is  the  prime  factor  in  this  decision-making  process.  Loss  of  accuracy  in 
geodetic  computations  negates  the  validity  of  any  study  being  performed.  If  such  studies  are  consistently 
performed  over  ranges  not  exceeding  100  nmi,  then  any  of  the  Earth  models  presented  here  is  adequate. 
However,  satellite  applications  with  appropriate  motion  cannot  be  included  in  any  flat  Earth  model 
approach.  Hence,  flat  Earth  models  must  be  excluded  from  further  considerations  in  solving  problems  that 
will  arise  in  the  1990s. 

The  WGS  system  is  an  Earth  model  that  offers  point  positioning  control  on  a  global  basis.  Although 
large  differences  may  exist  even  between  contiguous  datums,  such  differences  can  be  handled  uniformly. 
Spherical  Earth  models  offer  no  resolution  to  this  problem.  Consequently,  for  all  the  reasons  cited,  it  is 
recommended  that  the  ENEWS  Program  embark  on  enhancing  its  scenario  environment  generator  by  using 
the  DoD-wide  standard  WGS  model  to  perform  all  required  geodetic  calculations. 
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Appendix  A 

THE  GEODETIC  INVERSE  SOLUTION  SUBROUTINE 


This  appendix  shows  the  Geodetic  Inverse  Solution  subroutine  used  for  the  calculations  done  in  this 
report.  The  original  code  for  this  routine  was  obtained  from  the  Defense  Mapping  Agency  (DMA),  for 
which  the  authors  are  indebted. 

Essentially,  this  routine  has  as  input  the  latitudes  and  longitudes  of  the  two  points  O  and  A  on  the 
ellipsoid  defined  in  Fig.  1  in  the  body  of  this  report.  It  calculates  the  surface  or  arc  length  in  meters,  and  the 
O  to  A  and  AioO  bearings  in  seconds.  Fixed  constants  are  defined  or  generated  externally  to  the  routine  and 
passed  into  the  routine  in  the  “Ellipsoid”  common  block.  These  formulae  are  accomplished  with  double 
precision  accuracy  on  the  computer.  The  ellipsoid  definition  for  WGS-84  used  with  this  Inverse  Solution  is 
as  follows: 

A  =  6378137 
F  =  1.  /  298.257223563 
B  =  (  1  -  F  )  *  A 

The  following  constants  are  used  in  the  inverse  and  related  geodetic  routines  not  shown  here,  and  are 
calculated  once  during  initialization  for  optimum  efficiency. 

ASQR  =  A  *  A 

BSQR  =  B  *  B 

BSQOASQ  =  BSQR  /  ASQR 

BOA  =  B / A 

FF1  =  1  -  BOA 

F2  *  (  ASQR  /  BSQR  )  -  1 

FF16  =  FF1  /  16 

RTS  =  206264.806247 

The  code  used  for  the  Inverse  Solution  is  shown  in  the  following  text.  The  symbol  represents  a 
line  continuation.  Inputs  and  outputs  are  currently  given  in  seconds  for  angles  and  in  meters  for  distances. 
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SUBROUTINE  INVERSE ( OLATS , OLONS , OAAZ , ALATS , ALONS , AOAZ , ARC ) 

SUBROUTINE  INVERSE  CALCULATES  THE  ARC  LENGTH  BETWEEN  POINTS  A 
AND  THE  OBSERVER  AT  POINT  0,  THE  BEARING  FROM  POINT  A  TO 
POINT  0,  AND  THE  BEARING  FROM  THE  OBSERVER  TO  POINT  A. 

GEODETIC  INVERSE  COMPUTATION  BY  T.  VINCENTY  JAN  84 

INPUTS : 

A  -  SEMIMINOR  AXIS  IN  METERS 
B  -  SEMIMINOR  AXIS  IN  METERS 
OLATS  -  LATITUDE  OF  POINT  0  IN  SECONDS 

OLONS  -  LONGITUDE  OF  POINT  0  IN  SECONDS 

ALATS  -  LATITUDE  OF  POINT  A  IN  SECONDS 

ALONS  -  LONGITUDE  OF  POINT  A  IN  SECONDS 

OUTPUTS : 

ARC  -  SURFACE  ARC  LENGTH  BETWEEN  POINTS  A-0  IN  METERS 
OAAZ  -  BEARING  OF  POINT  A  FROM  POINT  0  IN  SECONDS 

AOAZ  -  BEARING  OF  POINT  0  FROM  POINT  A  IN  SECONDS 

IMPLICIT  REAL* 8 ( A-H , 0-Z ) 

REAL*8  LAM,  L,  K 

EQUIVALENCE  ( K , LAM ) 

COMMON  /ELLIPSOID/  A,  B,  ASQR,  BSQR ,  BSQOASQ ,  F,  F2 ,  FFl , 

&  FF16,  BOA,  RTS,  DPI,  TOLINV,  TOLDIR, 

&  SPHERENAME,  MAXSPHERES ,  NRSPHERE ,  NRSP, 

&  KLOOP 

REAL*8  A,  B,  ASQR,  BSQR,  BSQOASQ,  F,  F2 ,  FFl,  FF16 

REAL*8  RTS,  DPI,  TOLLOOP 

INTEGER*  4  MAXSPHERES,  NRSPHERE,  NRSP,  KLOOP 

CHARACTER* 2 2  SPHERENAME 


A 

B 

ASQR 

BSQR 

BSQOASQ 

FFl 

FFl  6 

F2 

BOA 

RTS 

TOLINV 

TOLDIR 

DPI 

SPHERENAME 

MAXSPHERES 

NRSPHERE 

NRSP 

KLOOP 


SEMIMINOR  EARTH  AXIS  (METERS) 

SEMIMINOR  EARTH  AXIS  (METERS) 

A  SQUARED 
B  SQUARED 

CONSTANT  =  B  SQUARED  OVER  A  SQUARED 
CONSTANT  =  1  -  F 
CONSTANT  =  FFl  /  16 

CONSTANT  =  (A  SQUARED  /  B  SQUARED)  -  1 
CONSTANT  =  B  OVER  A  =  B  /  A 
RADIANS  TO  SECONDS  CONVERSION  FACTOR 
INVERSE  PROGRAM  LOOP  LIMIT  TOLERANCE 
DIRECT  PROGRAM  LOOP  LIMIT  TOLERANCE 
DOUBLE  PRECISION  PI 

THE  NAME  OF  THE  CURRENT  SPHERE  SELECTED 
THE  MAXIMUM  NUMBER  OF  SPHERES  ALLOWED 
NUMBER  OF  THE  CURRENT  SPHERE  SELECTED 
THE  CURRENT  NUMBER  OF  SPHERES  DEFINED 
INVERSE  LOOP  COUNTER 
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DATA  TOLINV  /  l.D-12  / 

IF  (  ALATS  .EQ.  OLATS  .AND.  ALONS  . EQ .  OLONS  )  THEN 
ARC  =  0.D0 
AOAZ  =  0 . DO 
OAAZ  =  0.D0 
RETURN 
ENDIF 

UA  =  BOA  *  DS IN ( OLATS/RTS )  /  DCOS ( OLATS/RTS ) 

UB  =  BOA  *  DSIN( ALATS/RTS )  /  DCOS ( ALATS/RTS ) 

COSUA  =  1 . DO  /  DSQRT (  UA**2.D0  +  1.D0  ) 

SINUA  =  UA  *  COSUA 

COSUB  =  1 . DO  /  DSQRT (  UB**2.D0  +  1.D0  ) 

L  =  (OLONS  -  ALONS)  /  RTS 

LAM  =  L 

S  -=  COSUA  *  COSUB 

BAZ  =  S  *  UB 

FAZ  =  BAZ  *  UA 

KLOOP  =  0 

280  CONTINUE 

KLOOP  «  KLOOP  +  1 

IF  (  KLOOP  .GT.  100  )  THEN 

WRITE ( 14 , * )  '  LOOP  LIMIT  COSA  = ' , COSA 

WRITE (14,*)  '  OLATD  =  ',  OLATS  /  3600. 

WRITE (14,*)  '  OLOND  OLONS  /  3600. 

WRITE (14,*)  '  ALATD  ALATS  /  3600. 

WRITE (14,*)  '  ALOND  =',  ALONS  /  3600. 

GOTO  400 
ENDIF 

SINLA  =  DS IN ( LAM ) 

COSLA  =  DCOS (LAM) 

UA  =  COSUB  *  SINLA 

UB  =  -SINUA  *  COSUB  *  COSLA  +  BAZ 

SINT  =  DSQRT (  UA**2.D0  +  UB**2.D0  ) 

COST  =  S  *  COSLA  +  FAZ 
TH  =  DATAN2( SINT, COST) 

SINA  =  S  *  SINLA  /  SINT 
COSA  =  -SINA**2 . DO  +  1 . DO 
IF  (  COSA  .EQ.  0 . DO  )  THEN 
COSTM  =  COST 
ELSE 

COSTM  =  -2. DO  *  FAZ/COSA  +  COST 
ENDIF 

C  =  (  ( -3 . DO *COSA  +  4. DO)  *  FFl  +  4. DO)  *  COSA  *  FF16 
TEST  =  LAM 

LAM  =  ( ( (COSTM**2 .DO  *  2. D0-1. DO)  *  COST  *  C  +  COSTM)  * 
&  SINT*C+TH )  *  SINA  *  FFl  *  (1.D0  -  C)  +  L 

IF  (  DABS (TEST-LAM)  .GT.  TOLINV  )  GOTO  280 
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400  CONTINUE 

C  ****  CALCULATE  THE  BEARING  FROM  POINT  A  TO  THE  OBSERVER 

BAZ  =  DATAN2 ( -COSUA  *  SI NLA,  COSUB  *  SINUA  -  BAZ  *  COSLA  ) 
AOAZ  =  BAZ  *  RTS 

IF  (  AOAZ  .LT.  0 . DO  )  AOAZ  =  AOAZ  +  1.296D+06 

C  ****  CALCULATE  THE  BEARING  FROM  THE  OBSERVER  TO  POINT  A 

FAZ  =  DATAN2 ( UA , UB ) 

OAAZ  =  FAZ  *  RTS 

IF  (  OAAZ  .LT.  0.D0  )  OAAZ  =  OAAZ  +  1.296D+06 
K  =  F2  *  COSA 

C  =  ( ( (K*(-175.D0)+320.D0)*K-768.D0)*K+4096.D0) 

&  *  K/16384 . D0+1 .DO 

D  =  ( ( ( K* (  -47 . DO ) +  74 .DO ) *K-128 .DO ) *  K  +  256. DO)*  K/  1024. DO 


s  =  (  ( 

( 

COSTM* * 2 . DO  *  4. DO  - 

3. DO 

) 

& 

★ 

( 

SINT**2 . DO  *  4. DO  - 

3. DO 

) 

& 

★ 

COSTM  *  ( — D )  /  6. DO 

& 

+ 

( 

COSTM**2 . DO  *  2. DO  - 

1  .DO 

) 

& 

* 

COST  )  *  D  /  4. DO  + 

COSTM 

) 

& 

* 

SINT  *  ( — D )  +  TH 

ARC  =  S 

* 

C  *  B 

RETURN 

END 
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THE  GEODETIC  DIRECT  SOLUTION  SUBROUTINE 

The  Geodetic  Direct  Solution  subroutine  is  shown  in  program  form  on  the  next  two  pages.  The  “&” 
symbol  represents  line  continuations.  Inputs  and  outputs  are  given  in  seconds  for  angles  and  in  meters  for 
distances.  The  Direct  Solution  subroutine  calculates  the  latitude  and  longitude  of  an  end  point,  given  a 
beginning  point,  and  the  bearing  and  distance  to  the  end  point.  The  many  constants  are  defined  during  model 
initialization  as  discussed  in  Appendix  A.  These  constants  are  passed  to  the  routine  by  means  of  the 
ELLIPSOID. CDK  common  block  that  is  shown  in  the  listing. 

The  Direct  Solution  subroutine  also  contains  an  iterative  loop  like  the  Inverse  routine.  The  Direct 
Solution  routine  loop  begins  at  statement  label  10. 
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SUBROUTINE  DIRECT ( ALAT , ALON , BLAT , BLON , FAZ , BAZ , S ) 

C  GEODETIC  POSITION  COMPUTATION 
INPUTS: 

A  -  SEMIMINOR  EARTH  AXIS  (IN  COMMON) 

B  -  SEMIMINOR  EARTH  AXIS  (IN  COMMON) 

ALAT  -  POINT  A  LATITUDE 
ALON  -  POINT  A  LONGITUDE 
FAZ  -  FORWARD  BEARING  (POINT  A  TO  B ) 

S  -  SURFACE  DISTANCE 

OUTPUTS 

BLAT  -  LATITUDE  OF  POINT  B 
BLON  -  LONGITUDE  OF  POINT  B 
BAZ  -  BACK  BEARING  (  POINT  B  TO  POINT  A) 

IMPLICIT  REAL *8 ( A-H , O-Z ) 

REAL *8  LAM,  NUM 

C  ELLIPSOID  COMMON  BLOCK 

COMMON  /ELLIPSOID/  A,  B,  ASQR ,  BSQR ,  BSQOASQ ,  F,  F2 ,  FFl , 

&  FF16,  BOA,  RTS,  DPI,  TOLINV,  TOLDIR, 

&  SPHERENAME,  MAXSPHERES ,  NRSPHERE ,  NRSP, 

&  KLOOP 

EQUIVALENCE  (SINU,TANU),  ( AA, D ) ,  ( BB , LAM ) ,  (FIRST, NUM) 

DATA  TOLDIR  /  l.D-12  / 


COSAZ  =  DCOS (  FAZ  /  RTS  ) 

SINAZ  =  DSIN (  FAZ  /  RTS  ) 

TANU  =  (  DS IN( ALAT/RTS  )  /  DCOS (  ALAT/RTS )  )  *  BOA 
THA  =  DATAN2 ( TANU , COSAZ ) 

COSU  =  1 . DO  /  DSQRT ( 1 . DO  +  TANU**2.D0  ) 

SINU  =  TANU  *  COSU 

SINAL  =  COSU  *  SINAZ 

COSSAL=  1 . DO  -  SINAL* * 2 . DO 
XK  =  F2  *  COSSAL 

AA  =  ( ( (XKM-175.D0)  +  320. DO)  *  XK-768.D0)  *  XK+4096.D0) 

&  *  XK  /  16384. DO  +  1.D0 

BB  =  ( ( ( XK* (  -47. DO)  +  74. DO)  *  XK-128.D0)  *  XK+  256. DO) 

&  *  XK  /  1024. DO 

FIRST  =  S  /  B  /  AA 

TH  =  FIRST 
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10  COSTHM-  DCOS ( 2 . DO  *  THA  +  TH) 

SINTH  =  DSIN(TH) 

COSTH  =  DCOS ( TH ) 

TEST  =  TH 

TH  =  ( ( ( COSTHM*  *2 . DO  *  4. DO-3. DO) 

&  *  (  SINTH* * 2 . DO  *  4. DO-3. DO) 

&  *  COSTHM  *  (-BB)  /  6. DO 

&  +  ( COSTHM* *2 . DO  *  2. DO  -  1.D0)  *  COSTH  ) 

&  *  BB  /  4. DO  +  COSTHM)  *  SINTH  *  BB  +  FIRST 

IF  (  DABS ( TH-TEST )  . GT .  TOLDIR  )  GOTO  10 

TEST  =  SINU  *  SINTH  -  COSU  *  COSTH  *  COSAZ 
NUM  =  SINU  *  COSTH  +  COSU  *  SINTH  *  COSAZ 
DEN  =  BOA  *  DSQRT ( S INAL* * 2 . DO  +  TEST**2.D0) 

BLAT  =  DATAN2(NUM,DEN)  *  RTS 

LAM  =  DATAN2 ( SINTH*SINAZ , COSU*COSTH-SINU*SINTH*COSAZ ) 

D  =  ( (-3.DO*COSSAL+4.DO)*FFl+4.DO)  *  COSSAL  *  FF16 
^jjON  =  (  (  ( COSTHM*  *  2  .  DO  *  2. DO-1. DO)  *  COSTH  *  D  +  COSTHM) 
*  D  *  SINTH  +  TH) 

*  SINAL  *  FF1  *  (  D  -  1 . DO  )  +  LAM 
BLON  =  ALON  -  BLON  *  RTS 

IF  (  BLON  .GT.  . 648D06  )  BLON  =  BLON  -  .1296D07 
IF  (  BLON  .LT.-.648D06  )  BLON  =  BLON  +  .1296D07 
BAZ  =  DATAN2( SINAL, TEST)  *  RTS 
IF  (  BAZ  .LT.  0.D0  )  BAZ  =  BAZ  +  .1296D07 
RETURN 

END 
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Appendix  C 

CHORDAL  DISTANCE  CALCULATION  SUBROUTINE 

The  Chordal  Distance  subroutine  computes  the  chordal  distance  between  two  given  points  on  the 
Earth.  The  inputs  are  the  latitudes  and  longitudes  of  these  two  points  as  well  as  their  associated  elevations. 
As  with  the  other  geodetic  routines,  certain  fixed  constants  are  calculated  during  program  initialization  and 
passed  to  the  routine  by  means  of  the  ELLIPSOID. CDK  common  block. 
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SUBROUTINE  CHORD ( OLATR , OLONR , OELEV, ALATR , ALONR , AELEV , DCHORD ) 

C  SUBROUTINE  CHORD  CALCULATES  THE  CHORDAL  DISTANCE  BETWEEN 
C  POINT  0  (THE  OBSERVER)  AND  POINT  A  IN  DOUBLE  PRECISION  METERS. 

INPUTS: 

OLATR  -  LATITUDE  OF  POINT  0  IN  RADIANS 

OLONR  -  LONGITUDE  OF  POINT  0  IN  RADIANS 

OELEV  -  ELEVATION  OF  POINT  0  IN  METERS 

ALATR  -  LATITUDE  OF  POINT  A  IN'  RADIANS 

ALONR  -  LONGITUDE  OF  POINT  A  IN  RADIANS 

AELEV  -  ELEVATION  OF  POINT  A  IN  METERS 

OUTPUTS: 

DCHORD-  CHORDAL  DISTANCE  IN  METERS 
IMPLICIT  REAL*8 ( A— H , O-Z ) 

C  ELLIPSOID  COMMON  BLOCK 

COMMON  /ELLIPSOID/  A,  B,  ASQR ,  BSQR ,  BSQOASQ,  F,  F2 ,  FFl , 

&  FF16 ,  BOA,  RTS,  DPI,  TOLINV,  TOLDIR, 

&  SPHERENAME,  MAXSPHERES ,  NRSPHERE ,  NRSP,  KLOOP 

OC= ( ASQR ) /DSQRT ( ASQR* ( DCOS ( OLATR ) ) *  *2 . DO  +  BSQR* ( DSIN ( OLATR ) ) *  *2 . DO ) 

OX= ( OC+OELEV)*DCOS( OLATR )*DCOS( OLONR)  i  OBS  X-COORD 

OY» ( OC+OELEV ) *DCOS ( OLATR ) *DSIN( OLONR )  !  OBS  Y-COORD 

OZ- ( ( ( BSQOASQ ) *OC ) +OELEV ) *DSIN( OLATR )  !  OBS  Z-COORD 

AC=( ASQR) /DSQRT (ASQR* (DCOS (ALATR) )** 2 . DO+BSQR* ( DSIN ( ALATR ) ) **2.D0) 

AX«=(AC+AELEV)  *DCOS(  ALATR  )  *DCOS(  ALONR)  !  POINT  A  X-COORD 

AY=*  ( AC+ AELEV )  *DCOS  ( ALATR  )  *DSIN(  ALONR  )  !  POINT  A  Y-COORD 

AZ* ( ( ( BSQOASQ ) *AC ) +AELEV ) *DSIN ( ALATR )  !  POINT  A  Z-COORD 

DCHORD=»DSQRT  (  (  OX-AX )  *  *  2  .  DO  +  (  OY-AY  )  *  *  2  .  DO  +  (  OZ-AZ  )  *  *2  .  DO  ) 

END 
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Appendix  D 

FLAT  EARTH  MODEL  ANALYSIS 


Figures  DI  through  D3  give  the  range  error  plots  calculated  for  a  fixed  range  of  100  nmi.  In  Fig.  D1 
the  curves  for  the  Midpoint  and  End  Point  Flat  Earth  models  coincide  when  the  observer  is  at  the  equator. 
However,  in  Fig.  D2  the  End  Point  Flat  Earth  model  calculation  peaks  at  nearly  double  the  equatorial  error 
when  the  observer  is  at  45°N  latitude.  The  Midpoint  Flat  Earth  model  range  errors  are  reduced  in  Fig.  D2 
as  compared  to  those  in  Fig.  Dl .  At  northern  latitudes,  as  shown  in  Fig.  D3,  the  errors  are  exorbitant  for  the 
End  Point  Flat  Earth  model,  but  they  have  not  changed  much  in  peak  value  from  the  low-latitude  plots  for 
the  Midpoint  Flat  Earth  model.  They  are  approximately  0.5  nmi.  From  these  plots,  it  is  clear  that  the  End 
Point  Flat  Earth  model  range  calculations  increase  in  error  with  the  increasing  latitude  away  from  the 
equator. 

We  now  analyze  the  corresponding  bearing  errors  at  the  range  of  100  nmi.  When  the  observer  is  at  the 
equator,  as  in  Fig.  D4,  the  End  Point  Flat  Earth  model  errors  are  only  slightly  greater  than  the  midpoint 
model  errors.  Figure  D5,  with  the  observer  located  at  45°N  latitude,  shows  the  bearing  errors  coinciding  at 
peaks  of  approximately  0.85°  at  the  90°  and  270°  measurements.  However,  the  Midpoint  Flat  Earth  model 
values  exceed  those  for  the  End  Point  Flat  Earth  model  measurements  at  most  other  angles.  When  the 
observer  is  positioned  at  75°N,  as  shown  in  Fig.  D6,  this  effect  is  amplified  further.  Bearing  errors  near 
90°  are  excessive. 
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Fig.  D3  —  Flat  Earth  range  errors  vs  azimuth 
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Fig.  D4  —  Flat  Earth  bearing  errors  vs  azimuth 
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Range  100  nnru 

qo  Observer  Latitude  45°  N 

Observer  Longitude  0°  E 


End  Point  Flat  Earth  (  - ) 

180°  Midpoint  Flat  Earth  (  —  ) 


Fig.  D5  —  Flat  Earth  bearing  errors  vs  azimuth 
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Fig.  D6  —  Flat  Earth  bearing  errors  vs  azimuth 
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Appendix  E 

SPHERICAL  EARTH  MODEL  ANALYSIS 


Figure  El  shows  a  graph  of  the  selected  three  spherical  models  at  500  nmi.  The  peak  error  is  2.8  nmi 
when  using  the  Spherical  Approximation.  The  Navigation  Sphere  has  minimum  error  at  the  0°  and  180° 
azimuths  and  has  maximum  error  at  the  90°  and  270°  azimuths.  The  profile  of  the  range  errors  shown  in 
Fig.  E2  changes  when  the  500-nmi  range  is  used  at  45 °N.  The  Spherical  Approximation  is  predominantly 
better,  except  at  180°  where  the  Navigation  Sphere  is  better.  In  Fig.  E3,  the  range-error  curve  is  nearly 
circular.  The  Equal  Volume  sphere  has  minimum  error  over  the  northern  half  of  the  circle,  and  the 
Spherical  Approximation  has  least  error  over  the  southern  half.  The  Navigation  Sphere  is  consistently  worst 
case.  However,  all  spherical  models  are  within  0.5  nmi  in  magnitude. 

For  completeness,  errors  incurred  at  longer  distances  must  also  be  examined.  We  select  a  range  of 
5400  nmi.  This  represents  a  distance  across  a  quarter  of  the  Navigation  Sphere.  Figures  E4  through  E6 
show  three  latitudes  at  this  range.  In  Fig.  E4,  with  the  observer  at  the  equator,  the  Spherical  Approximation 
model  has  the  least  error,  peaking  near  6  nmi;  the  worst  error  is  exhibited  by  the  Navigation  Sphere  that 
peaks  near  9  nmi.  When  the  observer  is  moved  to  45°N,  Fig.  E5  shows  error  peaks  near  20  nmi,  and  the 
Navigation  Sphere  is  best  to  the  south  of  the  observer,  whereas  the  Spherical  Approximation  model  is  best 
north  of  the  observer.  When  the  observer  is  near  the  pole,  at  75°N,  as  shown  in  Fig.  E6,  the  errors  are 
comparatively  less,  peaking  at  9  nmi  north  of  the  observer  and  at  12  nmi  south  of  the  observer. 

To  examine  the  usefulness  of  the  spherical  models  at  even  larger  distances,  similar  calculations  were 
performed  at  a  range  of  10,000  nmi.  Figure  E7  indicates  that  the  Navigation  Sphere  is  worst,  and  the 
Spherical  Approximation  model  is  best  for  range  error  measurement  for  ranges  near  10,000  nmi  with  the 
observer  at  the  equator.  When  the  observer  is  moved  to  45°N  latitude,  as  shown  in  Fig.  E8,  errors  drop  and 
error  profiles  change.  With  the  observer  at  75°N,  as  in  Fig.  E9,  the  Spherical  Approximation  model  is 
consistently  the  worst  of  the  three  models,  peaking  at  9  nmi  and  the  Navigation  Sphere  is  optimum,  peaking 
at  an  error  of  only  3  nmi. 

Figures  E10  through  E12  show  bearing  errors  for  observer  latitudes  of  0°N,  45°N,  and  75°N  at  a 
range  of  5400  nmi.  Bearing  errors  for  the  purposes  of  comparing  spherical  models  are  not  useful,  since 
these  errors  from  all  spheres  tested  coincide,  as  can  be  expected.  However,  absolute  bearing  values  are  of 
interest.  The  peak  bearing  error  (0. 17°)  is  found  in  Fig.  El  1 ,  at  45°N,  whereas  maximum  errors  near  the 
equator  and  pole  are  slightly  under  0.1°.  In  all  these  cases,  bearing  errors  for  most  modeling  applications 
are  insignificant.  This  observation  is  also  true  for  a  10,000  nmi  range,  as  shown  in  Fig.  El  3.  The  maximum 
bearing  error  is  0.18°. 
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Fig.  E2  —  Spherical  Earth  range  errors  vs  azimuth 
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Range  500  nrru 

Observer  Latitude  75°  N 
Observer  Longitude  0“  E 


Spherical  Approximation  ( - ) 

180°  Navigation  Sphere  ( .  ) 

Equal  Volume  Sphere  (  ) 


Fig.  E3  —  Spherical  Earth  range  errors  vs  azimuth 
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Observer  Longitude  0°  E 


Spherical  Approximation  { -  ) 

180°  Navigation  Sphere  (  ) 

Equal  Volume  Sphere  (  ) 


Fig.  E4  —  Spherical  Earth  range  errors  vs  azimuth 


PICH  AND  LEROY 


Range  5400  nmi 

Observer  Latitude  45®  N 

Observer  Longitude  0°  E 
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Fig.  E5  —  Spherical  Earth  range  errors  vs  azimuth 
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Spherical  Approximation  ( -  ) 

180*  Navigation  Sphere  (  -  } 
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Fig.  E6  —  Spherical  Earth  range  errors  vs  azimuth 


38 


NRL  REPORT  9227 


Range  10000  nmi 

Observer  Latitude  0®  N 

Observer  Longitude  0°  E 


Spherical  Approximation  < - -  ) 

160®  Navigation  Sphere  { .  ) 

Equal  Volume  Sphere  (  ) 


Fig.  E7  —  Spherical  Earth  range  errors  vs  azimuth 


Range  10000  nmi 

Observer  Latitude  46*  N 
Observer  Longitude  0®  E 


Spherical  Approximation  ( - ) 

180*  Navigation  Sphere  ( .  ) 

Equal  Volume  Sphere  <  ) 


Fig.  E8  —  Spherical  Earth  range  errors  vs  azimuth 
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Range  10000  nmi  | 

Observer  Latitude  75®  N 
Observer  Longitude  0®  E 


Equal  Volume  Sphere  (  ) 


Fig.  E9  —  Spherical  Earth  range  errors  vs  azimuth 


Range  5400  nmi 
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Observer  Longitude  0®  E 


NOTE  All  curves  coincide 
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Fig.  E10  —  Spherical  Earth  bearing  errors  vs  azimuth 
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Fig.  El  1  —  Spherical  Earth  bearing  errors  vs  azimuth 


Fig.  El 2  —  Spherical  Earth  bearing  errors  vs  azimuth 
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Appendix  F 

COMPARISON  OF  FLAT  EARTH  MODELS  TO 
SPHERICAL  APPROXIMATION  MODEL 

Figure  FI  shows  the  0.5-nmi  maximal  error  also  shown  in  Fig.  D1  at  a  range  of  100  nmi  with  the 
observer  at  the  equator.  Similar  calculations  are  also  performed  by  using  the  Spherical  Approximation 
model  and  placed  on  the  same  graph  in  Fig.  FI .  For  these  close-in  distances  at  the  equator,  the  flat  Earth 
approximations  tend  to  yield  a  slightly  better  approximation  to  the  WGS-84  ellipsoid  than  does  the 
Spherical  Approximation.  For  example,  we  can  perform  range  calculations  as  far  out  as  200  nmi  before 
incurring  a  1-nmi  mile  error  with  either  flat  Earth  model  whereas  the  Spherical  Approximation  would  incur 
this  error  at  approximately  175  nmi.  However,  these  results  are  only  true  at  the  equator.  Figures  F2  through 
F7  provide  similar  plots  at  different  latitudes  on  an  equal  maximum  range  error  scale  for  ease  of 
comparison.  From  these  plots  it  is  clear  that  on  the  average  (that  is,  over  many  latitudes)  the  Spherical 
Approximation  outperforms  the  flat  Earth  models.  At  85°N  the  flat  Earth  models  deteriorate  rapidly. 

To  better  understand  the  characteristics  of  the  Spherical  Approximation  model  at  longer  distances, 
we  performed  tests  at  0°,  15°,  30°,  45°,  60°,  75°,  and  90°  north  observer  latitudes.  Ranges  from  10  to 
5400  nmi  were  selected,  and  plots  of  the  maximum  error  from  each  range  vs  distance  from  the  observer 
were  drawn.  Figures  F8  through  F14  show  the  resultant  plots.  When  the  observer  is  on  the  equator,  as  in 
Fig.  F8,  the  maximum  error  is  10.5  nmi,  occurring  at  a  range  of2900  nmi.  The  maximum  error,  21  nmi,  is 
shown  in  Fig.  FI  1  with  an  observer  latitude  of  45°N.  This  point  occurs  at  a  range  of  5400  nmi  from  the 
observer.  As  the  observer  moves  farther  north  in  latitude  (Figs.  F12  through  F14),  the  error  decreases. 
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Fig.  FI  —  Surface  range  errors  vs  surface  range 
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F5  —  Surface  range  errors  vs  surface  range 


Fig.  F7  —  Surface  range  errors  vs  surface  range 


Observer  Latitude  0°  N 


Fig.  F8  —  Spherical  Approximation  surface  range  errors  vs  surface  range 
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Observer  Latitude  15°  N 


Fig.  F9  —  Spherical  Approximation  surface  range  errors  vs  surface  range 


NRL  REPORT  9227 


Observer  Latitude  45°  N 


o-H- - 1 - 1 - 1 - r - i - 1 - r - i - t - j - 1 

0  500  1000  1500  2000  2500  3000  3500  4000  4500  5000  5500 

Distance  from  Observer  (nmi) 


Fig.  FI  1  —  Spherical  Approximation  surface  range  errors  vs  surface  range 
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Fig.  F12  —  Spherical  Approximation  surface  range  errors  vs  surface  range 
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Observer  Latitude  ?5°  N 


Fig.  F13  —  Spherical  Approximation  surface  range  errors  vs  surface  range 


Observer  Latitude  90°  N 


Fig.  F14  —  Spherical  Approximation  surface  range  errors  vs  surface  range 
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